The Internet, which started as a tool for the delivery of simple textual content, has now grown into a full-blown source of high-quality, multi-media content. With the incorporation of high-quality video and audio CODECs, and increased bandwidth over the Internet, high-definition video content can now be streamed to a user's personal computer for viewing. Content such as movies, television shows, commercials or the like are available to a user by simply accessing an appropriate URL or internet address through a player, such as a Flash Player. The video content is encoded at the audio video source end and then distributed through a streamer over one or more content delivery networks to one or more user devices. The user devices include a player which is configured to decode the stream and then render it on a display and speaker associated with or communicatively coupled to the user device.
Streaming content is basically multimedia content (i.e., audio, video, graphics, text or a combination of any of these) that flows in continuous pieces or packets, and is normally presented or rendered to an end-user while being delivered by a streaming provider. Thus the term “streaming” refers to the delivery method of the content rather than to the content itself. The distinction is usually applied to content that is distributed over telecommunications networks, as most other delivery systems are either inherently streaming (e.g., radio, television) or inherently non-streaming (e.g., books, video cassettes, audio CDs). The verb ‘to stream’ is also derived from this term, meaning to deliver media in this manner. Internet television is a commonly streamed media.
Attempts to display audio/video content on computers date back to the earliest days of computing in the mid-20th century. However, little progress was made for several decades, primarily due to the high cost and limited capabilities of computer hardware.
From the late 1980s through the 1990s, consumer-grade personal computers became powerful enough to display audio/video content. The primary technical issues related to the provision of audio/video content included:                having sufficient processing power and bus bandwidth to support the required data rates; and        creating low-latency interrupt paths in the operating system (OS) to prevent under running or emptying out the buffer thereby invoking a delay while additional content was obtained.        
Thus, as advances in technology addressed these issues, the provision of audio/video content was realized. Yet, despite the advances in technology, computer networks were still quite limited, and audio/video content was usually delivered over non-streaming channels, such as by downloading a digital file from a remote web server and then saving it to a local drive on the end user's computer or storing it as a digital file and playing it back from CD-ROMs.
During the late 1990s and early 2000s, Internet users saw great technological advancements in networking technology including:                large increases in network bandwidth, especially in the last mile;        increased access to networks, especially the Internet;        use of standard protocols and formats, such as TCP/IP, HTTP, and HTML; and        commercialization of the Internet.        
These advances in computer networking combined with powerful home computers and modern operating systems made streaming of audio/video content practical and affordable for ordinary consumers. Stand-alone Internet radio devices were developed to offer listeners a “no-computer” option for listening to audio streams.
In general, multimedia content has a large volume, so media storage and transmission costs are still significant; to offset this somewhat, media are generally compressed for both storage and streaming.
Increasing consumer demand for streaming of high definition (HD) content to different devices in the home has led the industry to develop a number of technologies, such as Wireless HD or ITU-T G.hn, which are optimized for streaming HD content without forcing the user to install new networking cables.
Although the present disclosure is focused on the delivery of video content, it should be appreciated what whenever, the term audio/video, video, or media is used, it should be implied to cover all instances of media type and combinations thereof unless explicitly limited to a particular media type.
A media stream can be streamed either live or on demand. Live streams are generally provided by a means called true streaming. True streaming sends the information straight to the computer or device without saving the file to a hard disk. On demand streaming is provided by a means called progressive streaming. Progressive streaming saves the file to a hard disk and then is played from that location. On demand streams are often saved to hard disks and servers for extended amounts of time; however, a live stream is typically only available at one time only (e.g. during a sporting event, show or speech). Streaming media storage size (in the common file system measurements megabytes, gigabytes, terabytes, and so on) is calculated from the streaming bandwidth and length of the media using the following formula (for a single user and file):storage size (in megabytes)=length (in seconds)×bit rate (in bit/s)/(8×1024×1024) where 1 mebibyte=8×1024×1024 bits.
As an example, one hour of video encoded at 300 kbit/s (a typical broadband video in 2005 and it is usually encoded in a 320×240 pixels window size) would be:(3,600 s×300,000 bit/s)/(8×1024×1024) or around 128 MiB of storage.
If the file is stored on a server for on-demand streaming and this stream is viewed by 1,000 people at the same time using a Unicast protocol, the bandwidth requirement is:300 kbit/s×1,000=300,000 kbit/s=300 Mbit/s of bandwidth
This is equivalent to around 135 GB per hour. Of course, using a multicast protocol the server sends out only a single stream that is common to all users. Hence, such a stream would only use 300 kbit/s of serving bandwidth.
Designing a network protocol to support streaming media raises many issues. These issues include the type of protocol to use versus the type of service and content to be delivered. Datagram protocols, such as the User Datagram Protocol (UDP), send the media stream as a series of small packets. This is simple and efficient; however, there is no mechanism within the protocol to guarantee delivery. It is up to the receiving application to detect loss or corruption and recover data using error correction techniques. If data is lost, the stream may suffer a dropout.
The Real-time Streaming Protocol (RTSP), Real-time Transport Protocol (RTP) and the Real-time Transport Control Protocol (RTCP) were specifically designed to stream media over networks. RTSP runs over a variety of transport protocols, while the latter two are built on top of UDP.
Reliable protocols, such as the Transmission Control Protocol (TCP), guarantee correct delivery of each bit in the media stream. However, they accomplish this with a system of timeouts and retries, which makes them more complex to implement. It also means that when there is data loss on the network, the media stream stalls while the protocol handlers detect the loss and retransmit the missing data. Clients can minimize this effect by buffering data for display. While delay due to buffering is acceptable in video on demand scenarios, users of interactive applications such as video conferencing will experience a loss of fidelity if the delay that buffering contributes to exceeds 200 ms.
Unicast protocols send a separate copy of the media stream from the server to each recipient. Unicast is the norm for most Internet connections, but does not scale well when many users want to view the same program concurrently. Multicast protocols were developed to reduce the data replication (and consequent server/network loads) that occurs when many recipients receive unicast content streams independently. These protocols send a single stream from the source to a group of recipients. Depending on the network infrastructure and type, multicast transmission may or may not be feasible. One potential disadvantage of multicasting is the loss of video on demand functionality. Continuous streaming of radio or television material usually precludes the recipient's ability to control playback. However, this problem can be mitigated by elements such as caching servers, digital set-top boxes, and buffered media players.
The present disclosure is focused on the delivery of live media to a user over the Internet. Several factors can easily diminish the enjoyment of a user's experience when viewing video content over the Internet. In general, due to data transmission errors, bandwidth constraint (especially during high-volume times), latency, etc., the delivery audio/video content to a user can be choppy or stuttered. From a user's perspective, this can become quite annoying. For instance, if the traffic delays are too severe, a playback of the content may be paused while additional content is received by the user device. To overcome such issues, the use of buffering has been employed. By using buffering, a sufficient amount of content can be delivered and queued up in a user device or distribution node such that at all times, the next segment of content is readily available to the user device. Thus, when a sufficient amount of content is buffered up for delivery to the user device, if the delivery of subsequent packets are delayed, the buffer makes this appear seamless to the user who is able to view a steady stream of content by simply accessing the content that has been stored in the buffer. The buffers are generally set to be sufficiently large to mitigate any latency issues, or at least the most common latency issues that would disrupt the delivery of the audio/video content.
In most settings, such as watching sporting events, lectures, television shows, movies, etc., buffering of the audio and/or video content is an acceptable practice to provide consistent and pleasurable viewing experiences. However, it will be appreciated that in such situations, the content being viewed is actually skewed from the real-time generation of the audio/video content. For instance, a live sporting event may be 3-5 seconds ahead in real-time from the actual audio/video content that being viewed by a user over the Internet. Such delays are not noticeable or realizable from the user's perspective unless the user is also at the live event.
However, in some situations, it will be appreciated that such delays may not be acceptable by a user. For instance, if the user expects or is relying on the provision of real time audio/video but, the actually delivery is delayed by a number of seconds, the user may be adversely affected. As an example, if a remote surgeon is participating in an operation procedure, a delay of a few seconds could result in the loss of a patient. Many other examples may come to mind in which the real-time, un-delayed delivery of audio/video content is required.
One of the leading providers of audio/video playback software applications for personal computers is the FLASH PLAYER marketed by ADOBE. The FLASH PLAYER includes the decoding functions necessary to receive an audio/video stream and then render to a user interface, such as a browser window, for viewing and listening. Generally, a personal computer equipped with a FLASH PLAYER may interface with a delivery system, such as a content delivery network (CDN) or some other infrastructure or service provider or FLASH based infrastructure to receive streamed content over a specific channel. Although exemplary embodiments are described as interfacing or operating in conjunction with a CDN, it will be appreciated that any public or private network, as well as other communication infrastructures may also be employed. The specific channel is typically identified through the use of a URL or URL along with codes, passwords, identifiers, etc. The audio/video content is usually buffered at one level by the CDN and, the audio/video content delivered to the user device is also usually buffered.
In operation, using the FLASH PLAYER as an example, a user can enter information to identify a specific stream (such as entering or selecting a URL) and a connection is then established between the CDN and the FLASH PLAYER and, the delivery of the audio/video content is begun.
The protocol defining the interface between the FLASH PLAYER and the CDN includes several commands that can be used to control the delivery of the audio/video content.
As is well known in the art by those familiar with programming of modules, applications, etc. that interface with or interact with FLASH PLAYERS, the following commands are included in the FLASH protocol:
Interactive Flash Commands
CommandFunctionPlayBegins or resumes the playback of the audio/video streamStopPauses the playback of the audio/video stream until a Playcommand is issued again.GoTo andThe seek command is used to designate the frame or locationStop (Seekin an audio/video stream to move forward or backwards to.Stop)For this command, the requested frame is loaded or moved tobut, the playback of the content is halted.GoTo andThe seek command is used to designate the frame or locationPlay (Seekin an audio/video stream to move forward or backwards to.Play)For this command, the requested frame is loaded and thecontent is buffered pre-roll and playback begins.
Using such commands, a playback application, such as FLASH PLAYER, can build additional controls for the access, delivery and playback of audio/video content. For example, the following value for Get URL may instructs a player to seek to the specified time in the presentation timeline:                command:seek(time)        
More specifically, the following command may instruct the player to seek to location 1:24.2 in a content timeline:                command:seek(1:24.2) where the time format could be defined as dd:hh:mm:ss.xyz with dd being days, hh being hours, mm being minutes, ss being seconds, x being tenths of seconds, y being hundredths of seconds, and z being milliseconds. Depending on the implementation, only some of the fields may be required. Further, if the time value does not include a decimal point, the player may read the last field as the seconds. For example, 1:30 could be interpreted as 1 minute and 30 seconds, whereas 1:30:00 could mean 1 hour and 30 minutes. Also, note that all of the following commands are equivalent with each seeking to the point 90 minutes into the audio/video content timeline:        command:seek(1:30:00.0)        command:seek(90:00)        command:seek(5400)        
Playing, Pausing, or Stopping content delivery
The following commands can cause a player to play, pause, or stop the presentation, respectively:                command:play( )        command:pause( )        command:stop( )        
Those skilled in the art will be familiar with other players and other software commands and controls for these players.
But, what is needed in the art is a player that is more suitable for the delivery of real-time audio/video content that is not delayed, or at least is minimally delayed.